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Introduction 


Memory problems have plagued humankind since time out of, well, memoiy. Don Norman, 
in Things that make us smart (Norman, 1993), has observed that one of the techniques that 
humans have used to improve their overall condition is the use of external aids and artifacts. 
These artifacts improve our ability to calculate, our ability to perform physical work, and our 
ability to remember. 

Most people have used memory aids of one form or another. Some use mnemonics or 
visualization to aid recall. It is often said that ancient storytellers would visualize walking through 
rooms or scenes in order to recall sections of verse or stoiy. Today we use external aids as well. 
Strings around the finger, sticky notes, pocket calendars and portable digital assistants (PDAs) all 
serve to keep information “at our fingertips.” These aids are extremely successful, but they 
require our attention. A calendar unchecked does little good. Many modem electronic memoiy 
aids have alerting devices. These alarms, bells, whistles and even synthesized vocalizations 
attempt to reach out and grab our attention so that we can attend to the memoiy item. 

These alerts are excellent at grabbing and redirecting our attention. However, it is often 
overlooked that they grab our attention away from something else. If that something else were 
just daydreaming, this might be acceptable. Quite often the something else is more meaningful 
and substantial. This interruption can range from a nuisance to a disaster. For example, we walk 
down the hall to pick up an important document, we are interrupted by our PDA announcing that 
we have to make a phone call, we immediately proceed to make the call and forget about the 
document. While one could argue that this is simply a matter of setting priorities, I would counter 
that alerts are often extremely compelling, especially to certain personality types (e.g., those who 
cannot stand to let a telephone ring) and that even the distraction of hearing the alert can 
sometimes sufficiently interfere with the ongoing task. In other words, the alerts themselves can 
set the priorities. 

We need to find ways to enhance our memoiy that do not interfere with ongoing tasks or 
other memory items. The overall goal of this research is to look at the efficacy of doing so using 
computers and other electronic devices to assist us. The assisting approach suggested here is 
called subliminal memoiy enhancement or, less controversially, Momentary Memoiy Priming 
(MMP). The concept of MMP is to briefly present material (target material) that we wish the 
person to remember on a display screen where that person is performing another activity. It is 
hypothesized that the brevity of the presentation will not disrupt the ongoing activity, but that the 
priming will be sufficiently perceptible as to reactivate, or refresh the memoiy trace of the target 
information in long-term memoiy. There are several application areas for this type of aid. One 
such application would be for prospective memoiy (remembering to do something in the future). 
A classic example of this problem is remembering to pick up an item at the store on the commute 
home from work. The task that the person is performing is driving and the target memoiy item 
that the person wishes to remember is to deviate from the current procedure (i.e., not go home but 
stop at the store). While prospective memoiy is perhaps the most intriguing application of such a 
memoiy aid, it can also be useful in remembering vital facts while performing another task - for 
example, remembering how to format a document while working on a word processor. This 
application would refresh procedural and declarative memory. Retrospective memoiy, that is, 
information that the person has already learned, could be refreshed just in time without having to 



leave the current task to look it up. A mechanic who is troubleshooting a malfunctioning system 
could benefit from refreshing his or her memoiy of the functional connections of the system. A 
pilot programming a complicated flight management system could benefit by a brief refresher on 
how to input a new waypoint on the route. This type of memoiy (retrospective memoiy) is the 
type that will be explored in this research. Previous research has demonstrated that this technique 
(MMP) might be effective (Wallace, et al 1991). But if it is effective, how robust is the effect? 

This study explored the boundaries of the effectiveness of MMP. MMP was used in a word 
recall task. The subject was asked to memorize a list of words presented sequentially on a video 
screen. This set of words was the target list. Then the subject was asked to memorize a second 
set of words presented in the same manner. This set of words was called the interference list. 
This interference task was designed to erase the target words from working memoiy so that they 
should only be present in long-term memory. For the treatment condition, the target list was 
momentarily primed (i.e., displayed for 33 msec.) during the presentation of the interference list. 
After assessing the recall of the interference list, the subject was requested to recall the target 
words (without seeing them again). Both recall measures were compared with a control condition 
where no priming occurred. This study represents a worst-case implementation for the MMP 
concept in that the secondaiy task (the interference task) is designed to directly interfere with 
memorizing the target words. It was thought that if the MMPs are effective, one would expect 
that the final recall of the target list would improve over the control condition whereas the recall 
of the interference list would be unaffected. If the recall for the interference list was poorer for the 
treatment condition, then the MMPs negatively affected the task of memorizing the interference 
list. If the recall of the target list was the same or worse in the treatment condition, then it would 
imply that the MMP does not have sufficient priming power to overcome the deleterious effects 
of the interference task. 

Chapter 2 presents a literature review of relevant topics in the area of memoiy research and 
enhancement, and subliminal presentation of information. Chapter 3 discusses the concept of 
MMP in relation to the research described in the preceding chapters. Chapter 4 describes the 
experimental hypotheses tested in this study and the methods and procedures. Chapter 5 presents 
the experimental results of the study. Chapter 6 provides a discussion of these results and their 
implications. Chapter 7 summarizes the study and discusses future work. 
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Literature review 


Memory 

Memory is often considered a single thing. Someone with good memoiy would be thought of 
as being able to recall most of the things that he or she experiences. However, Tulving (Endel 
Tulving, 2000) has argued that the word memoiy is often used to mean many different things. 
People use the word, memoiy, to mean the neurological functions of encoding and storing an 
experience for recall, the notional store (apart from the physical neurology) for information in the 
mind, the actual contents of that information, the cognitive processes leading up to and including 
the retrieval of some experience, and the human phenomenon of being aware of remembering 
something. Tulving called for using the word memoiy as an adjective, as in memory store, 
memoiy encoding, memoiy capacity, and so on. 

In this chapter, I will confine the discussion to the cognitive aspects of memoiy. While the 
exact definitions of the processes of memoiy are not shared by all researchers, there is a tentative 
consensus that the following processes are important to memory: attention to some event, 
encoding of the information about that event, storage/learning of that information, maintenance of 
that store, cueing for retrieval of the information, retrieval of the information, and expression of 
that information (Baddeley, 1998; Bower, 2000; Lezak, 1995; Lockhart, 2000; Shapiro & 
Eichenbaum, 1997). A person sees a phone number (attention), ‘memorizes’ that number by 
storing it in working memoiy as a auditoiy or visual item (encoding); attempts to transfer the item 
from working memoiy to long-term memoiy by developing a mnemonic for remembering it 
(storage); repeats it to himself over and over (maintenance), passes a phone (cueing), recalls the 
number using the mnemonic (retrieval), and dials the phone (expression). Failure in any one of 
these processes can result in the apparent failure of memoiy in general. It is often the case that a 
person may actually know the information but have difficulty recalling it or expressing it. 
Alzheimer’s disease often manifests itself in this way during the early stages. The patient knows 
the word, it is on the tip of her tongue, but she simply cannot bring it to mind. Cueing can 
sometimes assist these patients in recalling the word. In frying to recall the name of the current 
president, the cues ‘shrubbery’ (semantic) and ‘W’ (associative) might quickly enable the patient 
to say ‘Bush’ and to know that this is correct. 

While forgetting commonplace information is a manifestation of diseases like Alzheimer’s, it 
is certainly not indicative. Virtually eveiyone forgets things. Schacter (Schacter, 2001) describes 
a memory expert who has virtual recall of incredibly long lists yet cannot remember to keep 
appointments. Cases like this have led researchers to not only categorize the processes involved 
in memoiy, but also the types of information that are remembered. These types are usually 
represented in dichotomies: auditory/verbal vs. visio/spatial (Baddeley, 2000; Bower, 2000), 
declarative vs. procedural (Lezak, 1995; Shapiro & Eichenbaum, 1997), semantic vs. episodic 
(Baddeley, 1998; Bower, 2000; Schacter, Wagner, & Buckner, 2000), cognitive vs. motor 
(Baddeley, 1998; Bower, 2000; Jonides & Smith, 1997; Lezak, 1995; Shapiro & Eichenbaum, 
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1997), implicit vs. explicit (Baddeley, 1998; Bower, 2000; Brown & Craik, 2000; Graf & 
Schacter, 1985), and retrospective vs. prospective (Baddeley, 1998; Nallan, Kennedy, & 
Kennedy, 1991). These dichotomies are generally determined by deficits in certain individuals. 
For example, a person may forget the names of all her friends, relatives, and places that she has 
lived (declarative and cognitive memory), but still be able to play the piano (procedural and 
motor memory). It is apparent that these different types of memoiy information are processed 
(encoded, stored, and retrieved) in different ways (perhaps using different areas of the brain or 
using different encoding, storage, and retrieval schemes). 

This categorization of memory into six processes and six dichotomies allows researchers to 
specifically target their experiments to certain aspects of memoiy. There are 72 potential 
combinations of memoiy processes and information type (# of processes x 2# of dichotomies 1). 
The present study explores the following processes and types. The processes are memoiy storage, 
memoiy maintenance, memoiy cueing, and memoiy retrieval. The information type is 
auditory/verbal, declarative, semantic, cognitive, explicit, and retrospective. 

Momentary Memory Primes 

In an experiment designed to test the utility of subliminal information, Wallace, et al. 
(Wallace, Flaneiy, & Knezek, 1991) set out to determine if providing subliminal help messages 
could aid users of a text line editor. They created a new text editor, called PITA. It was similar to 
the Emacs text editor (Stallman, 1981), which is commonly used by computer programmers, in 
functionality yet all of the commands were different. Like Emacs, there was a help screen that 
could be called up that showed all of the commands and what they meant. Subjects (with previous 
experience with text editors like Emacs but not with PITA) were given a brief instruction 
regarding PITA and were then given a fde to edit and make corrections. The study explored the 
effect of presenting the help screen periodically for an exposure of approximately 1 7 ms on the 
text-editing screen. The independent variables of the experiment were display content - the help 
screen, garbage text in the same format as the help screen, or a blank screen - and presentation 
schedule - fixed interval of 15 seconds or fixed ratio based on the keystrokes made by the 
subject. The results showed no difference in task performance, that is, there were no differences 
in errors or amount of text corrected. There were significant differences in number of times, the 
frequency, and the ratio (of keystrokes to help calls) of times the subject brought up the help 
screen based on screen content. For example, the mean number of times that the subjects called 
up the help screen was 3.954 for the subliminal help screen presentation, 6.95 for the blank screen 
presentation, and 7.767 for the garbage presentation. Means for the average time between help 
calls (286.2s - help screen; 244.6s - blank screen; and 149.6s - garbage) and the average number 
of commands between requests for help (17.5 - help screen; 10.6 - blank screen; and 7.7 - 
garbage) reflect similar trends. Differences were only significant (p < .05) between the help 
screen presentation and the garbage presentation. There were no significant findings for the 
schedule (fixed or ratio) treatment nor the interactions between schedule and screen content. 

The results indicate that the subliminal presentation of the help screen assisted the subjects in 
remembering the system commands, so that they did not have to leave the current editing task to 
call up the help display. The results also indicate that the presentation of non-relevant information 
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(garbage) degraded memory performance in some way. There was no direct measure of memoiy 
(i.e., asking the subjects to list the command information after the test) so it is difficult to 
determine exactly how the displayed information improved or degraded the subjects’ recollection 
of the help screen. 

The term ‘subliminal’ usually evokes an affective response. Most people immediately think 
of the subliminal advertising and attempts to influence behavior (Dixon, 1971). The most 
compelling studies have been those involved with influencing the affect of an individual 
(Bornstein, Leone, & Galley, 1987; Krosnick, Betz, Jussim, & Lynn, 1992; Meyer & Walker, 
1999). In these studies, subjects were given subliminal stimuli that had either positive or negative 
emotional content and then asked to react to secondaiy stimuli. For example, Krosnick, et al. 
(Krosnick, Betz, Jussim, & Lynn, 1992) found that subject’s attitudes and beliefs about 
personality types regarding individuals whose pictures were shown as a secondaiy stimuli were 
affected by whether the subliminal stimuli, which were presented prior to the secondaiy stimuli, 
were emotionally positive or negative. Other studies have shown that when subjects are 
subliminally primed with geometric shapes, they rated those shapes as more likable or pleasant in 
a forced choice situation. In many ways, this resembles the false-fame effect (Jacoby, Woloshyn, 
& Kelley., 1989). An interesting study by Smith and Rogers (Smith & Rogers, 1994) that was 
designed to test whether subliminal messages affected selection behavior (that is, if a subliminal 
prime instructed the subject to choose one product over another, would the prime affect the 
subject’s choice?) The answer was no; however they found that material that contained the 
subliminal primes (in this case, a commercial) was actually less memorable. This suggests that 
providing information subliminally may actually disrupt memoiy. The subjects were not 
explicitly instructed to remember the commercials so this falls under the categoiy of implicit 
memoiy (see (Baddeley, 1998) for a definition of implicit memoiy). 

What constitutes a subliminal presentation? The standard definition is perception without 
conscious awareness, that is, the prime has some effect on the subject’s behavior, but the subject 
is never consciously aware of perceiving the prime (Bornstein & Pittman, 1992). Determining 
whether the subject is actually aware of the prime is controversial. The two basic methods of 
determining conscious perception are self-reports and discrimination measures based on 
conscious experience (Merikle & Reingold, 1992). Merikle states that self-report is often suspect 
as being unreliable (although it is regularly used in most all other fields of psychological, 
neurological, and medical research). Similarly, discrimination measures are often criticized as 
being incomplete, that is, as simply not testing the full range of conscious awareness and as 
thereby missing something. Merikle and Reingold (Merikle & Reingold, 1992) state that given 
these criticisms it would be virtually impossible to prove that anything was not subject to 
conscious awareness. They propose that a more useful metric would be the differences in 
sensitivities between direct and indirect measures. Direct measures are measures such as asking 
the subject to discriminate whether the prime was previously seen, and indirect measures are such 
as asking whether the subject favors or likes one prime over another (Merikle & Reingold, 1992). 
In this case you are not determining whether the stimulus is subject to conscious awareness, but 
the degree of conscious perception of the stimulus. Thus, a brief stimulus would rate a lower 
degree of conscious perception. 

If a subject is not consciously aware of subliminal priming, can it affect explicit memoiy, 
which by definition involves conscious effort? A study by Kamiya, et al. (Kamiya, Tajika, & 
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Takahashi, 1994) suggests that the answer is no. In these three experiments, subjects were 
exposed to masked word-stimuli for durations of 36 ms or 54 ms and asked to name the word if 
they could. After this segment of the experiment, they were given a word fragment completion 
test in which they were to fill in the first word that came to mind. Then they were given a 
recognition test where the subjects were given a list of words and asked to circle the words that 
were presented in the cueing segment. The authors felt that these tests were indirect measures of 
memory. However, the latter test is often used as a direct test of conscious perception (Kihlstrom, 
Barnhardt, & Tataiyn, 1992). The reason that this measure was considered to measure indirect 
memory is that the subjects were not instructed that they would be tested on recalling these words 
later - but this is generally used when the subjects have consciously perceived the stimuli 
(Merikle & Reingold, 1991). The results indicated that only the words that were named (i.e., 
consciously recognized and therefore not subliminal) had a significantly positive memoiy effect. 
The average results for two such experiments are presented in Table 1. There was no statistical 
difference in the word fragment completion task between the failure to recognize group and the 
new group, indicating that the subliminal words did not prime the subjects. 



Naming 


Test 

Success 

Failure 

New 

Word-fragment completion 

.68 

.35 

.32 

Recognition 

.72 

.18 



Table 1 Average Proportion Correct for experiments 1 & 2 for Correctly Named Items, Incorrectly Named 
Items, and Nonstudied Items (Karniya, Tajika, & Takahashi, 1994) 


The authors found that there was a significant main effect (p < .01) for word naming. They 
also found that there was a significant interaction between test and word identification (p < .05). 
They performed separate one-way ANOVAs on the results of each test group and again found a 
main effect for word naming on each test (Kamiya, Tajika, & Takahashi, 1994). There are 
several pieces of data and analyses that are not included in this paper that would be enlightening 
to the interpretation of the results. There is no listing of the number of choices possible for the 
word-fragment completion task. If there were only two for the fragment completion, then .50 
would be chance and the subjects would have scored less than chance (.35). If there were four, 
they would have performed better than chance. Similarly, the authors did not state how many 
words were on the recognition list. There is another important analysis that is missing - one-way 
ANOVAs for each of the Naming groups. If the numbers are comparable, it appears that the 
word-fragment completion is more sensitive to subliminal stimuli than more direct recognition 
tests, and this is what Merikle seems to call for (Merikle & Reingold, 1992). One important 
difference between the Kamiya study and the use of MMPs is that the words that were primed 
where not previously memorized. That is, it looked at placing the words in memory rather than 
reinforcing their trace in memory. 

Ionescu & Erdelyi (1992) performed several experiments addressing the ability to recall 
subliminal cues. They found that both words and pictures that were presented using a 
tachistoscopic projector could be recalled at better than chance levels for durations as brief as 5 
ms (the limit of their hardware setup). They argued that this was not truly subliminal since by 
their definition, subliminal meant not operating above chance. Using the 5 ms duration, they 
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reduced the illumination of the presentation by a factor of 3 and still found above chance levels. It 
was only when they reduced the illumination by a factor of 5 that statistically chance levels of 
recall were produced. In these experiments, the subjects were trying to read the words and 
pictures presented. In other experiments, such as (Merikle & Reingold, 1991), where subjects 
were not instructed to look at or recognize the stimuli, chance levels of recognition were obtained 
at much longer durations (430 ms in Merikle’s study). Attentional focus is clearly critical 
regarding whether a presentation is available to conscious perception. 

Attentional focus is also important to whether an item will be remembered. Simply being 
exposed to a stimulus, even at a high frequency and long duration, does not mean that the 
stimulus will be remembered. Berkerian and Baddeley (Bekerian & Baddeley, 1980) found that 
even though subjects had been passively exposed to a stimulus over 1,000 times, their recall was 
only 12 to 22% accurate. Similar results were obtained by Obermiller (Obermiller, 1985). This is 
consistent with the theory of levels of processing devised by Craik and Lockhart (Craik & 
Lockhart, 1972). This theoiy claims that the more deeply an item is processed, the better it is 
remembered. Thus, the memoiy of the word Dachshund will be more durable if the subject 
elaborates on the word by noting that it is a dog, and is often called a hot dog, and that he once 
knew of a Dachshund named Schnitzel, (i.e., semantic processing) than if he reasons that 
Dachshund rhymes with Datsun or ‘That soon’ (i.e. lexical processing). Transversely, the 
memoiy will be less durable if he simply repeats the word to himself over and over. The latter 
does not require a significant amount of processing where as the first requires the greatest amount 
of processing and elaboration. The former, deeper levels of processing are called elaborative 
rehearsal where as simply repeating the memoiy item over and over is called maintenance 
rehearsal. In the Wallace experiment, the participants supposedly did not recognize the 
presentations. Therefore, it is doubtful that the presentations provided any elaborative rehearsal. 
What is perhaps more likely is that the subjects’ memoiy was perceptually reinforced, as 
discussed in the following paragraphs. 

In the early stages of memoiy research, researchers held that there were two types of memoiy 
tests - a recall test and a recognition test - and that the latter was more sensitive than the former 
(Bower, 2000). However, Tulving and Thomson (E. Tulving & Thomson, 1973) showed that 
there were cases where information could be recalled but not recognized. In this experiment, 
subjects learned numerous word pairs. The words were common words that have a relatively high 
frequency of usage. Subjects were then given one of the two word pairs and asked to recall the 
other. Then they were asked to look through a list of common words and mark those that they 
recognized as having been presented earlier. If recognition is simply a more sensitive test of 
memoiy than recall, the subjects’ performance on the recognition test should have been better 
than the recall test. However, the opposite was true. The subjects were better able to recall than 
recognize when there was a cue present. 

Could the presentation of the HELP screen in the Wallace experiment have served this cueing 
purpose? Perhaps, however, the HELP screen contained far too many words to have been 
attended to in the brief presentation for a single word to cue the entire screen. What is more likely 
is that the perceptual form of the HELP screen cued the perceptual pattern, which in turn 
refreshed the memoiy trace of the PITA commands. (Since the garbage information in the 
Wallace experiment was presented in a similar format, it would appear that this cueing is highly 
sensitive and that errors in the cue (that is, differences from the actual cue) can actually reduce 
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memory performance rather than enhance it.) This form of priming draws on the Perceptual 
Representation System (PRS) as described by Tulving and Schacter (Endel Tulving & Schacter, 
1990). In this system, the trace that is being reinforced is not a semantic or lexical trace but rather 
a pictorial or perceptual trace. Tulving and Osier (Endel Tulving & Osier, 1968) found that when 
two words were paired for memorization such as ‘CITY -dirty’, and subjects were cued with 
‘CITY’, they would respond with ‘dirty’. However, when cued with semantically equivalent 
words such as ‘VILLAGE’ or ‘TOWN’, they were not able to recall dirty. 



Momentary Memory Primes: An approach to memory enhancement 


The concept behind MMP is to reinforce the target memoiy trace of an individual while that 
individual is performing a secondaiy task. The theoiy of MMP depends on the following 
assumption - that the individual is attending to the visual channel and that the MMP can be 
presented on the visual channel. MMPs use the visual channel to present target information as 
opposed to other input channels. This is due to the fact that research in the effectiveness of 
auditory subliminal presentation of material has been inconclusive at best (Moore, 1995; 
Beyerstein & Eich, 1993). 

There has been little research in the area of tactile, olfactory, or gustatoiy presentation of 
subliminal stimuli and it is uncertain as to how these presentations could effectively be made. 
Thus the visual channel was selected as the presentation channel for MMPs. The target memory 
items must be presentable in a visual form. If the reinforcement of the memoiy trace is due to 
perceptual cues rather than lexical or semantic cues (which is suspected) then the original 
presentation of the target memoiy items must be visual as well, and the MMP should faithfully 
represent that visual presentation. That is to say, perceptual qualities (such as shape and font) 
should be preserved in the MMP presentation. If the memoiy trace reinforced is perceptual, it 
would not be effective if the original memoiy item was a picture of a house and the MMP 
presentation was the word ‘HOUSE’. Note that using the visual channel does not require that the 
MMPs are presented using the secondaiy task. It may be that the MMPs can be presented on the 
individual’s eyeglasses or even projected directly on the retina. 

Given this assumption, MMP may be a viable option for memoiy enhancement. The MMP 
would present target memoiy information in the subject’s fovea for brief duration at certain 
intervals during the performance of the secondaiy task. There are several research questions that 
must be addressed to determine the efficacy of this concept. 

Research Questions 

Duration and Contrast. 

As mentioned above, the presentation duration of so-called subliminal stimuli has varied 
greatly from experiment to experiment. In some experiments, the presentation duration has been 
specifically tailored to the individual subjects. Generally in these cases, presentations were given 
at shorter and shorter intervals until the subject could not name the presented item at levels better 
than chance. Durations at chance were considered subliminal. In other experiments, contrast 
against background was also manipulated. For contrast, the background shading was modified to 
make the presented item more or less readable. See figure 1 for an example of the contrast 
manipulation. Again, contrasts where recognition performance was at chance or below were 
considered subliminal. 
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Item 

High Low 

Contrast Contrast 

Figure 1 : Examples of High and Low Contrast Presentations 

The problem with tailoring duration and contrast to the individual is that there is no evidence 
that these levels are stable over time or variations in content. In the extreme case the levels would 
have to be reestablished frequently, which would be a time-consuming task in itself. The 
important attribute of MMP is not that it is subliminal or not consciously registered but that it 
does not interfere with the secondary task. This is potentially dependent on the relationship 
between the secondary task and the content of the MMP. Tailoring of the duration seems to be an 
uneconomical option. In this research I believe that a single, standard duration and contrast 
should be settled upon, rather than tailoring them to the individual, the secondaiy task, or the 
content. Thus, two of the main research questions of this area are: “what durations and contrasts 
are acceptable for enhancing memoiy of the target memoiy items for a varied population and 
information type?”; and “what durations and contrasts are acceptable for not degrading 
performance in a secondaiy task?” These answers will not likely be the same and a compromise 
will likely be necessaiy, trading off memoiy efficacy against task disruption. 

Schedule. 

Similar to duration and contrast, the frequency at which the MMPs are presented is an 
important issue. Too frequent presentations may distract from the secondaiy task, and too 
infrequent presentations may not be of benefit to the memoiy enhancement. While Wallace et al. 
found no influence of the schedule type (fixed interval or fixed ratio) on performance in their 
experiment; their results may not be generalizable. Therefore, two other important research 
questions are: “what schedules (both frequency and type) are acceptable for enhancing memoiy 
of target memoiy items?”; and “what schedules (both frequency and type) are acceptable for not 
degrading performance in a secondaiy task?” 

Relevance. 

The Wallace experiment demonstrated that while providing a meaningful cue improved 
performance over a control of a blank screen, a cue of irrelevant information could be detrimental 
to performance. This leads to an important question: “what do we mean by relevance?” In the 
Wallace experiment, the relevant information contained instructions on how to operate the text 
editor. But what about information that is only relevant to the user? What if information is 
important to the user but has nothing to do with the secondaiy task? Would it still be disruptive? 
For example, if a person is trying to remember a list of things to buy while driving, would this be 
a candidate for MMP in the driving task, or would the MMP have to have something to do with 
driving (for example, remembering to go to the store instead of going home)? 

Recall. 

In the Wallace experiment, recall was not explicitly measured, but inferred from the number 
of times the subject called up the HELP page. The subliminal cue was presented at (or near) the 
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time of recall (or so it was assumed). That is, when the subject needed the information, the 
subliminal cue was there. But will this kind of memory enhancement assist for future recall? As 
mentioned earlier, it would be interesting to test the subjects’ ability to recall the contents of the 
HELP screen after the task. If not, then it might be inferred that the cues only assisted in 
knowledge retrieval from long-term memory into working memoiy and did not enhance the 
quality (e.g., retention, veridicality, ease of recall, ease of recognition) of long-term memory. 

Retrospective and Prospective Memory. 

The Wallace experiment tested retrospective memoiy, that is, recalling what one had already 
tried to remember. In retrospective memoiy, it is usually the case that the individual is seeking the 
information (e.g., how to delete a word in the PITA text editor.) The cues simply aided a search 
that had already begun. In prospective memoiy (i.e., remembering to remember) the key is the 
cue itself. Can MMPs provide such cues or will the individual have to remember to remember 
(e.g., “What was it that I was supposed to do?”)? 

These represent just a few of the questions that must be explored in assessing the efficacy of 
MMPs. The conditions that could be addressed are as follows: 

Prospective Commission (e.g., does the subject remember to perform the task in the 
future?) 

Retrospective Content (e.g., can the subject recall, recognize, or be cued into recalling 
past information?) 

Recall During Secondaiy Task (e.g., can the subject multitask/hold two pieces of 
information?) 

Recall After Secondaiy Task (e.g., can subject recall information after an interference 
task?) 

Relevant to the Secondaiy Task (e.g., will the subject use the MMP information now?) 

Relevant to the ‘target’ Task (e.g., will the subject use the MMP information later?) 

Irrelevant to the Subject (e.g., will the subject remember MMP information that is not 
useful to him or her?) 

The experiment described in this paper only addressed a subset of these questions. A portion 
of the experimental design space that is defined by these questions is portrayed in table 2. The 
shaded cells represent the conditions that were tested in the Wallace experiment. The cells with 
italicized text represent the conditions that were tested in this study. 
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Tense 

Retrospective/ 

Prospective 

Type 

Commission/ 

Content 

Task 

Recall/ 

Recognition/ 

Cued 

Time 

During ST / 
After ST 

Relevance 

Relevant-ST / 
Relevant-I / 
Irrelevant 

Schedule 

Fixed Ratio / 
Fixed Interval / 
Variable Ratio / 
Variable 
Interval 

Retrospective 

Content 

Recall 

During 

Relevant-ST 

Ratio 

Retrospective 

Content 

Recall 

During 

Relevant-ST 

Interval 

Retrospective 

Content 

Recall 

During 

Relevant-I 

Ratio 

Retrospective 

Content 

Recall 

During 

Relevant-I 

Interval 

Retrospective 

Content 

Recall 

During 

Irrelevant 

Ratio 

Retrospective 

Content 

Recall 

During 

Irrelevant 

Interval 

Retrospective 

Content 

Recall 

After 

Relevant-ST 

Ratio 

Retrospective 

Content 

Recall 

After 

Relevant-ST 

Interval 

Retrospective 

Content 

Recall 

After 

Relevant-I 

Ratio 

Retrospective 

Content 

Recall 

After 

Relevant-I 

Interval 

Retrospective 

Content 

Recall 

After 

Irrelevant 

Ratio 

Retrospective 

Content 

Recall 

After 

Irrelevant 

Interval 

Retrospective 

Content 

Recognition 

During 

Relevant-ST 

Ratio 

Retrospective 

Content 

Recognition 

During 

Relevant-ST 

Interval 

Retrospective 

Content 

Recognition 

During 

Relevant-I 

Ratio 

Retrospective 

Content 

Recognition 

During 

Relevant-I 

Interval 

Retrospective 

Content 

Recognition 

During 

Irrelevant 

Ratio 

Retrospective 

Content 

Recognition 

During 

Irrelevant 

Interval 

Retrospective 

Content 

Recognition 

After 

Relevant-ST 

Ratio 

Retrospective 

Content 

Recognition 

After 

Relevant-ST 

Interval 

Retrospective 

Content 

Recognition 

After 

Relevant-I 

Ratio 

Retrospective 

Content 

Recognition 

After 

Relevant-I 

Interval 

Retrospective 

Content 

Recognition 

After 

Irrelevant 

Ratio 

Retrospective 

Content 

Recognition 

After 

Irrelevant 

Interval 


Table 2 Partial Representation of Experiment Design Space 
ST - Secondary Task, I - Individual 
Shaded -Wallace Experiment, Italics - this study 


For the experiment described in this paper, the retrospective content was the focus, but 
instead of recall during the task, recall of information after the task was addressed. In addition, 
rather than the target information being relevant to the secondary task, the MMPs provided 
information relevant to the individual but not relevant to the secondary task. In fact, it provided 
information that was potentially disruptive to the secondary task. 

The experiment described here exposed subjects to a list of target memory items that they 
were asked to memorize. Then the subjects were shown a second list of memoiy items (i.e., the 
interference task). Subjects in the treatment condition saw a brief presentation of the target 
memoiy items presented for approximately 33 ms (MMP) during the presentation of the 
interference items. These MMPs were relevant to the subject (because they were asked to 
remember them) but were irrelevant to the interference task (i.e., memorizing the second set of 
words). Subjects in the control condition were presented with at brief blank screen of 
approximately 33ms during the interference task. The subjects were tested on both memoiy lists 
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by asking them to recall each list and to select (i.e., recognize) those words from a larger list of 
words. 

These conditions represent an extreme case for the use of MMPs. The information must be 
recalled after the secondary task (whereas the Wallace experiment used the information at the 
same time it was primed). This means that the MMP is strictly reinforcing long-term memory as 
opposed to retrieving information from long-term memoiy. Also, the secondaiy task is peculiar in 
that it is designed to deliberately stop attempts to reinforce long-term memoiy for the target 
items. This was not true in the Wallace experiment. In the Wallace experiment, the information 
was needed and used by the subject, which should have caused more elaborate processing 
regarding the target item. This elaborative processing should have also reinforced the memoiy 
trace in long-term memoiy. In this experiment, the information is not needed nor is it wanted at 
the time because the subject is dying to learn other words. If the MMPs were effective in this 
extreme case, then its effect is robust and therefore applicable to many applications. 

The following hypotheses are used to test the effectiveness of the MMP in this experiment. 

Hypothesis 1 

Subjects receiving the MMPs will have better recall for the target memoiy items than those 
subjects in the control condition. 

Hypothesis 2 

Subjects receiving the MMPs will have better recognition of the target memory items than 
those subjects in the control condition. 

Tentative Hypothesis 

A desirable hypothesis to test is the null hypothesis, that is that the MMPs will not adversely 
affect the interference task performance; that is, there will be no statistical difference in 
interference memoiy item recall or recognition between subjects in the control condition and 
those in the treatment condition. However, this would require proving the null hypothesis and this 
is not a generally accepted test. 
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Method 


Participants 

A total of 143 students from Virginia Commonwealth University were used in this 
experiment. Subjects had 20/20 corrected vision based on self-report. The subjects were 
recruited using the VCU Experimetrix program, which allows students in Psychology 101 classes 
to earn extra credit by participating in experiments. The subjects earned credit for a 1.5 hour 
experiment. No attempt was made to balance gender in the experiment. 99 of the subjects were 
female. The number of subjects was selected to insure power. The pre-test power analysis is 
discussed in Appendix F. The need for high power comes from the potential support of the third, 
tentative hypothesis stated above. 

Subjects participated in the experiments in groups of approximately 30. 

Apparatus 

The visual presentation was developed using Apple Computer’s Final Cut Pro (V3.0) 
software and was recorded on DVD. The stimuli were projected onto a screen using a video 
projector with a resolution of 1024 x 768 pixels per inch. 

Experimental Protocol 

This experiment was approved by the VCU Institutional Review Board for human subject 
protection and assigned the experimental protocol number 3762. The memoiy tasks chosen for 
this experiment were modeled after the Rey Auditory Visual Learning Test (RAVLT) (Schmidt, 
1996). The RAVLT is a memoiy test containing five learning trials for a list of 15 words, 
followed by an interference trial of 15 different words. The subject is then tested for recall of the 
15 words from the original trials. While generally applied using verbal cues and recalls, it has 
been shown to be effective in visual learning tasks as well. Appendix A provides an overview of 
the RAVLT. 

The subjects began by receiving a brief description of the experiment, followed by an 
opportunity to ask questions. They were then asked to sign a voluntary consent form (see 
Appendix C). The subjects were told that they were performing a memoiy experiment that was 
addressing the issue of screen distractions. They were told that during some segments of the test, 
the screen would flicker with different distractions. They were instructed that they were to ignore 
these distractions and concentrate on the memory task. This instruction was given to avoid 
biasing the subjects that the MMPs were present to aid their memoiy. It was considered to 
provide a conservative bias against the success of the MMPs since they were deliberately told to 
ignore them. After signing the consent form, they were given a brief format demonstration where 
they saw a typical presentation but did not have to remember the words. In place of actual words, 
sample words were used in this demonstration e.g., ‘wordl’, ‘word2’, etc. This procedure was an 
attempt to avoid undesirable interference in the actual memoiy task. 
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All subject responses were made on a paper worksheet that was covered after each trial. 

Each subject group received two mini-RAVLT sessions. Each session was either the 
treatment condition (MMP) or the control condition. The order in which the subjects saw each 
condition was counterbalanced (See Table 3a-f for counterbalance description). For the MMP 
condition, the subjects were presented a list of 15 words (i.e., target list), presented one at a time 
for a duration of one second with a one second presentation of a blank screen in between each 
word presentation. They were then given one minute to write down any of the fifteen words that 
they saw in any order. They were instructed to turn the paper over when they had finished. They 
were instructed not to use the time (that is, any time left over in the minute) remaining to continue 
memorizing the words. After the time was up, all papers were turned over. They then were shown 
the same presentation of 15 target words and given another minute to write down these words, 
again in any order. The subjects then received a third viewing of the target words and were asked 
to write them down again. The subjects were reminded that they would be tested on the target 
words later on in the experiment. This constituted the learning phase of the target memory items. 
The subjects were not presented with MMPs during this phase for either the control or treatment 
conditions. They were then given a new list of 15 different words (i.e., interference list) presented 
in the same format. However, for approximately 33 ms (i.e., one video frame) of each one-second 
interference word presentation, the screen displayed the target memory words (all 15 target 
words, one per interference word presentation) (figure 2). Thus in the MMP condition, the 
subjects saw each of the target words presented once again for approximately 33 ms. The subjects 
were given one minute in which to write down the interference memory words, again in any 
order. Finally, the subjects were given two minutes in which to write down the target memory 
words (without viewing them again). This completed the MMP session. 




500 msec 
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Figure 2: Interference presentation for Treatment Condition 


In the control session, two new sets of words were used for target and interference sets. The 


15 


session proceeded for the three learning trials in the same way as the MMP session. During the 
interference learning trial, the 33ms gaps (where the MMP had been placed in the MMP 
condition) contained only a blank display (figure 3). Thus, in this case, the word on the display 
screen appeared to blink white for a moment. The subjects were then asked to recall the 
interference set of words and then asked to recall the target set (from this session). 


500 msec 33 msec 467 msec 

Interference Control Interference 





V 


J V. 


V 


J V. 


V 


J 


1 second 1 second 1 second 


Figure 3: Interference presentation for Control Condition 


With so many lists of words being presented, the lists were described to the subject as “List 
1”, “List 2”, “List 3”, and “List 4”, which represented the target list for the first condition, the 
interference list for the first condition, the target list for the second condition, and the interference 
list for the second condition, respectively. 

Finally, when both sessions were completed, each subject was presented with a paper list of 
120 words (see appendix D), 60 of which had been viewed in the two sessions (15 target and 15 
interference for the MMP condition, and 15 target, and 15 interference for the control condition.) 
The subjects were asked to place an X by any word that they had seen during any of the 
presentations. They were given five minutes for this task. 

At the end of the experiment, the subjects were given a questionnaire (see appendix E) to fill 
out regarding subjective aspects of the experiment and then were excused. Figure 4 presents a 
chronological flow diagram of the experimental tasks that the each subject performed. 
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Figure 4: Experiment flow 


Experimental Conditions and Run Definitions 

The words used in this study were selected from the list of 116 five-letter words having a 
Thorndike-Lorge frequency (Thorndike & Lorge, 1972) of at least 50 per million, and were at a 
third-grade reading level. Appendix B provides the list of these words. The words were presented 
in all capitals and in Courier font (a non-kemed font) so that they took up the same horizontal and 
vertical screen space. 

In an attempt to avoid potential confounding issues from specific word lists (i.e., that one list 
of words was more memorable than another), two complete sets of words were randomly selected 
from the Thorndike-Lorge list. Table 3a-f shows the word lists and the counterbalancing that was 
used for the subject pool. Thus, order and word list were counterbalanced between subjects. 
Unfortunately, as will be described in the results sections below, the number of subjects that 
volunteered for the word list counterbalance (sessions 3 and 4) was nearly half of the original set 
(sessions 1 and 2). Order and word list were between-subject variables. Since each subject 
received each of the two conditions, treatment is a wi thin-subject design. 
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Session 1 - 26 Subjects 

Control First 

Treatment (MMP) Second 

Target 

Secondary 

(Interference) 

Target 

Secondary 

(Interference) 

List 1 

List 2 

List 3 

List 4 

AGAIN 

REACH 

CARRY 

STATE 

BEING 

MONEY 

SLEEP 

CLOSE 

MONTH 

STAND 

ROUND 

WOMAN 

THANK 

THERE 

BRING 

THREE 

SOUND 

LIGHT 

HEAVY 

BEGIN 

WHICH 

SINCE 

PLAIN 

EVERY 

COLOR 

LARGE 

SWEET 

BLACK 

FRONT 

DEATH 

CROSS 

LEARN 

VOICE 

DRIVE 

SERVE 

THINK 

LEAVE 

QUICK 

OFTEN 

THOSE 

FRESH 

PLANT 

FLOOR 

RIGHT 

ALONG 

WOULD 

WATCH 

RAISE 

FOUND 

TODAY 

DRINK 

HEART 

NEVER 

LAUGH 

WHILE 

STONE 

START 

NIGHT 

SHALL 

EARTH 


Table 3 (a) Session 1 words and order 


Session 2-23 Subjects 

Treatment ( 

:MMP) First 

Control Second 

Target 

Secondary 

(Interference) 

Target 

Secondary 

(Interference) 

List 3 

List 4 

List 1 

List 2 

CARRY 

STATE 

AGAIN 

REACH 

SLEEP 

CLOSE 

BEING 

MONEY 

ROUND 

WOMAN 

MONTH 

STAND 

BRING 

THREE 

THANK 

THERE 

HEAVY 

BEGIN 

SOUND 

LIGHT 

PLAIN 

EVERY 

WHICH 

SINCE 

SWEET 

BLACK 

COLOR 

LARGE 

CROSS 

LEARN 

FRONT 

DEATH 

SERVE 

THINK 

VOICE 

DRIVE 

OFTEN 

THOSE 

LEAVE 

QUICK 

FLOOR 

RIGHT 

FRESH 

PLANT 

WATCH 

RAISE 

ALONG 

WOULD 

DRINK 

HEART 

FOUND 

TODAY 

WHILE 

STONE 

NEVER 

LAUGH 

SHALL 

EARTH 

START 

NIGHT 


Table 3 (b) Session 2 words and order 
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Session 3-14 Subjects 

Control First 

Treatment (MMP) Second 

Target 

Secondary 

(Interference) 

Target 

Secondary 

(Interference) 

List 3 

List 4 

List 1 

List 2 

CARRY 

STATE 

AGAIN 

REACH 

SLEEP 

CLOSE 

BEING 

MONEY 

ROUND 

WOMAN 

MONTH 

STAND 

BRING 

THREE 

THANK 

THERE 

HEAVY 

BEGIN 

SOUND 

LIGHT 

PLAIN 

EVERY 

WHICH 

SINCE 

SWEET 

BLACK 

COLOR 

LARGE 

CROSS 

LEARN 

FRONT 

DEATH 

SERVE 

THINK 

VOICE 

DRIVE 

OFTEN 

THOSE 

LEAVE 

QUICK 

FLOOR 

RIGHT 

FRESH 

PLANT 

WATCH 

RAISE 

ALONG 

WOULD 

DRINK 

HEART 

FOUND 

TODAY 

WHILE 

STONE 

NEVER 

LAUGH 

SHALL 

EARTH 

START 

NIGHT 


Table 3 (c) Session 3 words and order 


Session 4-16 Subjects 

Treatment ( 

[MMP) First 

Control Second 

Target 

Secondary 

(Interference) 

Target 

Secondary 

(Interference) 

List 1 

List 2 

List 3 

List 4 

AGAIN 

REACH 

CARRY 

STATE 

BEING 

MONEY 

SLEEP 

CLOSE 

MONTH 

STAND 

ROUND 

WOMAN 

THANK 

THERE 

BRING 

THREE 

SOUND 

LIGHT 

HEAVY 

BEGIN 

WHICH 

SINCE 

PLAIN 

EVERY 

COLOR 

LARGE 

SWEET 

BLACK 

FRONT 

DEATH 

CROSS 

LEARN 

VOICE 

DRIVE 

SERVE 

THINK 

LEAVE 

QUICK 

OFTEN 

THOSE 

FRESH 

PLANT 

FLOOR 

RIGHT 

ALONG 

WOULD 

WATCH 

RAISE 

FOUND 

TODAY 

DRINK 

HEART 

NEVER 

LAUGH 

WHILE 

STONE 

START 

NIGHT 

SHALL 

EARTH 


Table 3 (d) Session 4 words and order 
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Session 5-33 Subjects 

Control First 

Treatment (MMP) Second 

Target 

Secondary 

(Interference) 

Target 

Secondary 

(Interference) 

List 5 

List 6 

List 7 

List 8 

SMALL 

EARLY 

UNDER 

WOMAN 

PAPER 

FRESH 

SOUTH 

HEAVY 

REACH 

THOSE 

COULD 

BLACK 

LAUGH 

SWEET 

CAUSE 

WORLD 

BUILD 

THANK 

RIVER 

BEING 

TABLE 

READY 

STAND 

DRINK 

ABOUT 

AGAIN 

PIECE 

RIGHT 

WHERE 

MONEY 

HOUSE 

WATER 

EARTH 

PLAIN 

VOICE 

UNTIL 

THERE 

THREE 

NIGHT 

ORDER 

CARRY 

FIRST 

EVERY 

SHORT 

FLOOR 

ROUND 

POWER 

DEATH 

MONTH 

COVER 

WATCH 

SIGHT 

AFTER 

VISIT 

FOUND 

QUICK 

STONE 

ABOVE 

TODAY 

POINT 


Table 3 (e) Session 5 words and order 


Session 6-31 Subjects 

Treatment ( 

:MMP) First 

Control Second 

Target 

Secondary 

(Interference) 

Target 

Secondary 

(Interference) 

List 7 

List 8 

List 5 

List 6 

UNDER 

WOMAN 

SMALL 

EARLY 

SOUTH 

HEAVY 

PAPER 

FRESH 

COULD 

BLACK 

REACH 

THOSE 

CAUSE 

WORLD 

LAUGH 

SWEET 

RIVER 

BEING 

BUILD 

THANK 

STAND 

DRINK 

TABLE 

READY 

PIECE 

RIGHT 

ABOUT 

AGAIN 

HOUSE 

WATER 

WHERE 

MONEY 

VOICE 

UNTIL 

EARTH 

PLAIN 

NIGHT 

ORDER 

THERE 

THREE 

EVERY 

SHORT 

CARRY 

FIRST 

POWER 

DEATH 

FLOOR 

ROUND 

WATCH 

SIGHT 

MONTH 

COVER 

FOUND 

QUICK 

AFTER 

VISIT 

TODAY 

POINT 

STONE 

ABOVE 


Table 3 (f) Session 6 words and order 
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Dependent measures 


The following dependent variables were analyzed based on the answers provided by the 
subjects. The differences between the number of words recalled on the third trial of the target 
memory list and the fourth trial (after the interference list trial) were computed. These differences 
represent the amount of memoiy loss over the interference trial. They reflect how many words 
the subject forgot over the interference task and is the primaiy determinant of the effectiveness of 
the MMP. The absolute number of words recalled from the target memoiy list on the last (fourth) 
trial represents a weaker measure but was, nonetheless recorded and analyzed. The number of 
words recalled from the interference memory list during that trial was used to compare the two 
conditions for interference and distraction due to the MMPs for the target words. For the 
recognition task, the mark made by the subject (i.e., X for recognized, blank for not presented) 
was recorded for each word and compared with whether the word was presented. The 
questionnaire items were also recorded. 

Analyses 

Hypothesis 1. 

Subjects receiving the treatment (MMP) for the target memoiy items will have better recall 
for the target memoiy items than those subjects in the control condition (i.e., that did not receive 
the treatment (MMP)). To test this hypothesis, a paired t-test was conducted on the differences 
between trials 3 and 4 for the target memory word list for the treatment (MMP) condition and the 
control condition. Also, a paired t-test was conducted on the absolute number of target words 
recalled at the end of each condition (i.e., the fourth recall of the target words). 

Hypothesis 2. 

Subjects receiving the treatment (MMP) will have better recognition of the target memoiy 
items than those subjects that did not receive the treatment (MMP). For this hypothesis, a paired 
t-test was conducted comparing the number of target words recognized in the recognition trial for 
the two conditions. 

Tentative Hypothesis. 

The treatment (MMP) will not adversely affect the interference task performance; that is, 
there will be no statistical difference in interference memoiy item recall or recognition between 
subjects in the control condition and those in the treatment condition. Paired t-tests on both the 
recall (i.e., recall trial in the interference task) and recognition tests were conducted between the 
two conditions of interference words. 


21 



Experimental Results 


All descriptive statistics and paired samples T-test analyses were performed using SPSS 
Standard Version 11.0.2 (15 April 2003) for the Macintosh. The significance level was set at 
95%. 14 subjects were removed from the data due to abnormal memoiy loss (>7 or <- 2), or 
abnormal pattern of improvement (nonmonotonic progressions greater than 2). Thus the n for the 
analysis was 129. 

Hypothesis 1 - Treatment (MMP) improved recall over Control 

There was no significant difference between treatment (M = 1.77 words, SD = 1.75) and 
control (M = 2.06 words, SD = 1.82) conditions for the memoiy loss measure (that is, the number 
of words correctly recalled in the fourth recall of the target words of the session subtracted from 
the number of words correctly recalled in the third recall). There was a statistically significant 
difference (p<.01) between the treatment and control conditions for absolute number of words 
correctly recalled in the fourth recall. This difference favored the treatment condition (M = 10.81 
words, SD = 2.80) over the control condition (M = 10.07 words, SD = 2.83). 

Hypothesis 2 - Treatment (MMP) improved recognition over Control 

There was no significant difference between treatment (M = 12.51 words, SD = 2.29) and 
control (M = 12.31 words, SD = 2.23) conditions for the number of words recognized at the end 
of the study. 

Tentative Hypothesis - Treatment (MMP) did not affect with interference task 
recall and recognition over control. 

There was no significant difference between recall of interference words for treatment (M = 
6.73 words, SD = 2.18) and recall of interference words for control (M = 6.76 words, SD = 2.02) 
conditions. Similarly, there was no significant difference between recognition of secondaiy 
(interference) words for treatment (M = 6.91 words, SD = 2.53) and recognition of secondaiy 
(interference) words for control (M = 7.12 words, SD = 2.87) conditions. 

However, questionnaire results indicate that many of the subjects felt that the MMPs or the 
Blank flickers (control condition) were annoying (see figure 5) and that they impaired memoiy 
(see figure 6). 
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Figure 5: Frequency of responses to the statement “The distractions were annoying.” 



Figure 6: Frequency of responses to the statement “The distractions made it harder to remember the second 

set of words.” 


Additional Observations 

Several ad hoc analyses were conducted to try to understand the findings above and prepare 
for future research. 
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Order effect. 

A one-way analysis of variance was performed on all of the dependent variables using order 
as the factor. The following variables were found to differ significantly based on order: The 
fourth recall of the target in the control condition F(l, 128) = 4.14, p<.05; the memory loss 
(difference between third and fourth recall) in the control condition F(l, 128) = 12.73, p<.01; 
recognition of the target words in the treatment condition F(l, 128) = 6.42, p<.05; and recognition 
of the interference words in the treatment condition F(l, 128) = 9.04, p<.01. Paired sample T- 
tests were performed comparing session order (first and second) regardless of wordlist. 

For the treatment first - control second order, significant differences were found in the fourth 
recall for the target words - p<.01, with M = 10.74 for the treatment condition (SD = 2.46) and M 
= 9.54 for the control condition (SD = 2.81); the memoiy loss over the interference task - p<.01, 
with M = 1.82 for the treatment condition (SD = 1 .62) and M = 2.64 for the control condition (SD 
= 1.74); and the recognition of the interference words - p<.01, with M = 6.23 for the treatment 
condition (SD = 2.83) and M = 7.39 for the control condition (SD = 3.23). 

For the control first - treatment second order, significant differences were found in the 
second recall for the target words - p<.05, with M = 10.97 for the treatment condition (SD = 
2.39) and M = 10.43 for the control condition (SD = 2.00); and the recognition of the target words 
- p<.01, with M = 12.99 for the treatment condition (SD = 1.92) and M = 12.22 for the control 
condition (SD = 2.34). 

The combined assessment of these data suggest the differences in fourth target word recall 
and the memoiy loss over the interference trials are likely to be due to decrements in the control 
condition (due to being near the end of the experiment) rather than improvements in the treatment 
condition. Similarly, the difference in the increase in the treatment target recognition where the 
treatment words are shown last is most likely attributable to the recency effect (Baddeley, 2000). 
The fact that the treatment negatively affected the interference recognition in the treatment first 
condition may be indicative of the novelty of the MMP. 

Word lists. 

For the target memoiy tasks, there were three sets of 15 words used for the treatment 
condition- the CARRY set, the AGAIN set, and the UNDER set (see tables 3 a — f). 
Respectively, there were three sets used for the control condition - the AGAIN set, the CARRY 
set, and the SMALL set. A one-way analysis of variance was conducted over the three word lists 
used for the treatment condition. The analysis showed that the choice of word list did have a 
significant effect on memoiy performance. Specifically, the first recall of the target words in the 
treatment condition - F(l, 128) = 6.94, p<.01; the second recall of the target words in the 
treatment condition - F(l, 128) = 7.81, p<.01; the third recall of the target words in the treatment 
condition - F(l, 128) = 3.34, p<.05; the first recall of the target words in the control condition - 
F(l, 128) = 6.43, p<.01; the second recall of the target words in the control condition - F(l, 128) 
= 8.38, p<.01; the third recall of the target words in the control condition - F(l, 128) = 11.11, 
p<.01; the fourth recall of the target words in the control condition - F(l, 128) = 11.85, p<.01; 
recognition of the target words in the treatment condition F(l, 128) = 4.12, p<.05; recognition of 
the target words in the control condition F(l, 128) = 19.56, p<.01; and recognition of the 
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interference words in the control condition F(l, 128) = 8.27, p<.01. 

Comparisons of the individual word list groups were performed using paired sample t-tests. 
For the treatment = CARRY - control = AGAIN group (n = 47), significant differences were 
found in the second recall for the target words - p<.05, with M = 10.51 for the treatment 
condition (SD = 2.08) and M = 9.79 for the control condition (SD = 1.85); the third recall for the 
target words - p<.01, with M = 12.28 for the treatment condition (SD = 1.95) and M = 11.15 for 
the control condition (SD = 2.56); the fourth recall for the target words - p<.01, with M = 10.47 
for the treatment condition (SD = 2.77) and M = 8.62 for the control condition (SD = 2.88); the 
memory loss over the interference task - p<.05, with M = 1.81 for the treatment condition (SD = 
1.77) and M = 2.53 for the control condition (SD = 2.00); the recognition of the target words - 
p<.05, with M = 11.77 for the treatment condition (SD = 2.59) and M = 11.09 for the control 
condition (SD = 2.66); and the recognition of the interference words - p<.01, with M = 6.74 for 
the treatment condition (SD = 2.34) and M = 5.94 for the control condition (SD = 2.52). In all 
cases, (statistically significant or not) the mean favored the treatment condition. 

For the treatment = AGAIN - control = CARRY group, which was the counterbalance to the 
previous group (n = 26), significant differences were found in the recognition of the target words 
- p<.01, with M = 12.96 for the treatment condition (SD = 1.66) and M = 14.04 for the control 
condition (SD = 1.55); and the recognition of the interference words - p<.01, with M = 6.46 for 
the treatment condition (SD = 3.06) and M = 8.50 for the control condition (SD = 2.37). In all 
cases, (statistically significant or not) the mean favored the control condition. The reader should 
note that the results of these two groups seems to indicate that the Cany word list is more 
memorable. 

For the treatment = UNDER - control = SMALL group (n = 56), there were no significant 
differences between the treatment and control conditions. The means for all cases (statistically 
significant or not) were mixed between favoring the treatment and control conditions. 

As a further exploration, a one-way analysis of variance was performed on the recalls without 
regard to treatment (i.e., first recall = first recall treatment + first recall control) with word list as 
the factor. Results were significant for all of the data except the interference words and the 
memory loss. The results are as follows: first recall F(3, 257) = 8.94, p<.01, second recall F(3, 
257) = 11.86, p<.01, third recall F(3, 257) = 9.41, p<.01, fourth recall F(3, 257) = 9.19, p<.01), 
target recognition F(3, 257) = 3.32, p<.05, and interference recognition F(3, 257) = 3.89, p<.05. 
Table 4 shows the sample number, the means and the standard deviations for each of the recalls. 
Figures 7 through 1 1 show the plots of the means for the four recalls of the target set and the 
recognition of the target set. 
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Recall 

Word List 

Subject 
Number (N) 

Mean 

Standard 

Deviation 

First 

Carry 

73 

6.96 

1.55 

Again 

73 

6.89 

1.55 

Under 

56 

8.16 

1.85 

Small 

56 

7.79 

1.73 

Second 

Carry 

73 

10.37 

1.99 

Again 

73 

9.89 

1.90 

Under 

56 

11.75 

2.00 

Small 

56 

11.20 

1.76 

Third 

Carry 

73 

12.25 

1.72 

Again 

73 

11.48 

2.16 

Under 

56 

13.07 

1.80 

Small 

56 

12.93 

1.94 

Fourth 

Carry 

73 

10.48 

1.87 

Again 

73 

9.14 

2.26 

Under 

56 

11.45 

2.23 

Small 

56 

11.09 

2.03 

Target 

Recogniti 

on 

Carry 

73 

12.58 

2.44 

Again 

73 

11.75 

2.51 

Under 

56 

12.93 

2.14 

Small 

56 

12.54 

1.53 

Interferen 

ce 

Recogniti 

Carry 

73 

7.37 

2.48 

Again 

73 

6.12 

2.72 

Under 

56 

7.27 

2.41 

Small 

56 

7.46 

3.00 


Table 4 Means and Standard Deviations of each recall and 
recognition based on word list. 
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Figure 7: Means for the first recall of the target set for each word list (n=258) 



Figure 8: Means for the second recall of the target set for each word list (n=258) 
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Figure 9: Means for the third recall of the target set for each word list (n=258) 



Figure 10: Means for the fourth (after the interference task) recall the target set for each word list (n=258) 
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Figure 1 1 : Means for recognition of the target set for each word list (n=258) 

Questionnaire Data. 

The questions dealt with the MMPs and the blanks that were presented during the 
interference word list presentation. The means for each question tended to be to the right of the 
midpoint (that is, towards disagree). The mode data was very interesting. The mode values for 
the statements “The distractions were annoying” and “The distractions made it harder to 
remember the second set of words” were both 2, indicating Agree. The mode values for “I tried 
to read the distractions,” “I was able to read the distractions,” “I noticed that some of the 
distractions were previous words,” and “The repetition of previous words in the distractions 
helped me remember them” were all 5, indicating strong disagreement. In hindsight, these 
answers may have been biased by the negative connotation of the word ‘distraction’ and by the 
fact the subjects were instructed not to pay attention to the distractors (MMPs). The histograms for 
the first two questions are presented in figures 5 and 6 above. The histograms for the remaining 4 
questions are found below in figures 12 through 15. 
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Distractions 

were 

Annoying 

Interfered 

with 

Memorizing 

Interference 

Tried to 
read 

distractions 

Able to 
read 

distractions 

Noticed 

distractions 

were 

previous 

words 

Distractions 

helped 

target 

memory 

Mean 

2.8 

2.7 

3.2 

3.1 

3.3 

3.9 

Mode 

2 

2 

5 

5 

5 

5 

St. Dev 

1.33 

1.28 

1.49 

1.45 

1.57 

1.24 


Table 5 


Descriptives for Questionnaire Data 
1 = Strongly Agree 5 = Strongly Disagree 



Figure 12: Frequency of responses to the statement “I tried to read the distractions.” 
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Figure 13: Frequency of responses to the statement “I was able to read the distractions.” 
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Figure 14: Frequency of responses to the statement “I noticed that some of the distractions were previous 

words.” 
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Strongly Agree Neutral Disagree Strongly 
Agree Disagree 


Figure 15: Frequency of responses to the statement “The repetition of previous words in the distractions 

helped me remember them.” 

The final part of the questionnaire asked the subjects to share any memorization strategy that 
they used. 77 subjects stated that they used some form of story or sentence making. 39 of the 
subjects stated that they used some sort of word association. 25 of the subjects stated that they 
used repetition. 16 of the subjects stated that they used to order of the words to help them 
remember. 9 stated that they chunked words into groups. 7 said that they used rhyming 
strategies. 4 subjects stated that they used visualization or imagery to help them remember. 1 
subject stated that he keyed on odd words and another stated that she translated the words into 
another language. Note that many of the subjects listed two or more strategies. 21 subjects did not 
answer the strategy question. 

Comparisons with the meta-norms for the RAVLT. 

One reason for using the RAVLT as a model for the experimental tasks in this experiment is 
the existence of established norms. The meta-norms for the first three trials in the RAVLT for 
ages 20 - 29 are listed in Table 6 and are taken from (Schmidt, 1996). While no age data was 
recorded for the subjects, most were ‘college age’, which generally ranges from 19 to 25. 


Trial 

Mean 

Standard Deviation 

1 

7.0 

1.8 

2 

9.9 

2.2 

3 

11.5 

2.1 


Table 6 Meta-norms for the age group of 20 to 29 year-olds for the RAVLT 


One-sample t tests were performed to compare these norms to the data for both the control 
and treatment conditions. In all but one case (control words for the first trial) the means for the 
recalls in this study were significantly higher (p < .05 for the comparison of the third trial for the 
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control, and p < .01 for the remainder) than the meta-norms. A possible explanation for these 
differences could be the fact that the words were visually displayed and were written down when 
recalled whereas the traditional RAVLT is generally oral in both presentation and recall. In 
seeing the words the subjects may have utilized both the visuo-spatial sketchpad and the 
phonological loop (as they said the words to themselves) for encoding the words (Baddeley, 
2000) whereas the RAVLT would only utilize the phonological loop. In writing down the words, 
they utilized more aspects of the brain (e.g., motor cortex). This more elaborative processing 
(Brown & Craik, 2000) may have reinforced the memoiy traces of the target words more than in 
the traditional RAVLT. Another possible explanation is that the demographics for the subjects in 
this test were substantially different from the ones used in the meta-norms. Finally, again it may 
be that the word lists used in this study were more memorable because they were all five-letter 
words where the number of letters in the words for the traditional RAVLT varies from 4 to 6 
letter words. 
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Discussion of Experimental Results 


The results do not find any evidence that the use of MMPs assists in recall or recognition of 
the target words after an interference task. The one finding that was significant - namely that 
subjects recalled a greater number of words for the fourth recall in the treatment condition 
compared to the control condition - is better explained by the apparent confounding of the word 
lists used themselves. If the word lists were equally memorable, one would expect that the third 
recall for all word lists to be statistically the same since the treatment had not been administered 
at that point. However, the post hoc analysis of the word lists demonstrate that this is not the case 
and that some lists were more memorable than others. The smaller number of subjects in the 
counterbalanced conditions (sessions 3 and 4) apparently caused the counterbalance to be 
ineffective. This was further confirmed by the fact that there was no significant difference in 
sessions 5 and 6 where entirely new word lists were used. 

Why didn’t the MMPs work to assist memoiy? To answer this we need to examine the 
differences between this study and the Wallace experiment. As mentioned in section 4 above, the 
conditions in this experiment were conservatively biased against the success of the MMPs in 
order to determine the robustness of effect. There are a number of possible explanations for their 
failure. One explanation is that mere exposure to the words does not enhance memoiy - 
regardless of the exposure time. As mentioned earlier, studies by Berkerian and Baddeley (1980) 
and Obermiller (1985) would suggest that mere exposure does not aid memoiy. In the Wallace 
experiment, the subjects were not only exposed to the presentation, it is inferred that they were 
seeking this information. Another explanation is that the interference task did its interfering quite 
well. Since the MMPs are subtle by design, the interference task may have overwhelmed any 
positive value. The secondaiy task in the Wallace experiment (the editing task) was not at odds 
with the primes. Yet another reason for these results is that in the Wallace experiment, the 
primed information was used at the time of priming. The results of this experiment would tend to 
suggest that the value of that prime is extremely short lived. Finally, the MMPs were in direct 
opposition to the secondaiy task and the subject was actively (per instructions and by dying to 
concentrate on the interference words) trying to ignore the MMPs. 

In hindsight, the assumption that the MMPs might be equivalent to a fourth session using the 
RAVLT (as discussed in the Appendix F: Preliminary Power Analysis) was inappropriate. In the 
fourth session of the RAVLT, the subject is deliberately dying to remember the words when they 
are exposed to them and then tries to recall the words. Both of these acts can tend to reinforce 
memoiy through elaborative processing (Craik and Lockhart 1972). Therefore expectations that 
the MMP exposure would be similar to the fourth presentation of the RAVLT were unrealistic. 

It is likely that these factors - 1) that mere exposure does not necessarily reinforce memoiy, 
2) that the secondaiy task was designed to disrupt the veiy memoiy that the MMPs were dying to 
reinforce, 3) that the half-life of the MMP value is veiy short, 4) that the primed information was 
not needed at the time of priming, and 5) that the primed information was not only irrelevant to 
the secondaiy task, it was in opposition to it - contributed to the failure of the MMPs to assist in 
remembering the target words. In addition, since many of the subjects reported that the MMPs 


34 



(and their blank control equivalents) were annoying and since many of the subjects claimed that 
were able to read the words presented, it may be that the single video frame (33ms) may be too 
long or insufficiently masked to provide a an unobtrusive prime. 


Conclusion 

“I have not failed 700 times. I have not failed once. I have succeeded in proving that those 700 ways 
will not work.” 

—Thomas Edison 


The Wallace experiment results indicate that there is potential value in using briefly displayed 
reminders of previously studied information to assist memoiy. The study described in this thesis 
indicates that this method of memory enhancement is not helpful across all types of memorization 
and recall tasks. This study also demonstrated that the disruption of one task by the MMP could 
be subjectively annoying to the participant (although there was no difference in performance). 
The Wallace experiment primed information that was useful to the subject while it was being 
presented and which the subject both desired and used at the time of presentation. The study 
described here primed information that was not useful to the subject when it was being presented 
and was not used at the time of presentation. It may be the MMPs do not refresh a memoiy trace 
but instead cue the recall of a previous memory (E. Tulving & Thomson, 1973). Thus the brief 
exposure to the MMP would only serve as a cue — what Tulving and Schacter (Endel Tulving & 
Schacter, 1990) call the Perceptual Representation System. 

Future studies in the area of MMPs might look at the use of information at the time of 
priming and information that is either relevant to the secondaiy task or not. I propose two such 
experiments. Both would involve negotiating a maze from a first person perspective. The map of 
the maze would be shown to the subject prior to moving in the maze and the subject would 
negotiate the maze a number of times on paper (the learning phase). Then the subject would be 
placed in a virtual first-person perspective (similar to many video games). As in the Wallace 
experiment, the subject could switch screen modes bring up the map (hiding the perspective 
view). However in the treatment condition, the subject would be primed with the map using 
MMPs while negotiating in the perspective mode. Anticipated benefits are the reduction of time 
to negotiate the maze, fewer wrong turns, and fewer calls for the map. This differs from the 
Wallace experiment in that the information presented is more directly related to the task - an 
analogous study using the Wallace paradigm would prime the editing directions, not the help 
screen. The second experiment would be similar to the first but would provide information that is 
not relevant to the secondaiy task. The information that would be primed would be similar to the 
memory words used in this task. The user would have to recall the words while negotiating the 
maze. At the end of each test sessions the subject would try to recall / recreate either the maze or 
the word list. 

Another set of experiments could look at the possibility of MMPs being used for prospective 
memoiy. Again, the maze scenario could be used. In this case, the subject would be given a task 
to remember to perform either after a particular time or after a particular juncture in the maze. 
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The prospective task would not be critical (otherwise, if it were critical then the reminder would 
be a more salient - supraliminal - presentation). The expected benefits would be that the MMPs 
would aid the subjects in remembering to perform the task but would be less disruptive to the 
maze-negotiating task than traditional alerts. 

The experiment described in the thesis demonstrates that the MMP concept has its limits and 
is not brutally robust. Having established a lower bound, it is time to explore less extreme and 
more amenable uses for MMPs. Information technology has great potential for helping humans 
remember information as long as designers remember to use it effectively. Less subtle methods 
than MMPs can lead to critical disruptions of ongoing tasks. There needs to be an effective 
balance in between salience and subtlety. I hope that these studies will lead to the discoveiy of 
that balance. 
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Appendix A 

Overview of the Rey Auditory and Verbal Learning Test 

Title: Rey Auditory and Verbal Learning Test (RAVLT) 

Author: Andre Rey (original French version); Edith Meyer Taylor (English adaptation) 

Publisher: Western Psychological Services, 12031 Wilshire Blvd. Los Angeles, CA 90025-1251 

Copyright Information: The test itself carries no copyright or trademark and administration of the test 
is not restricted. There are handbooks and administration sheets that are provided by the publisher which 
hold a copyright. There are no qualification restrictions on the materials. 

Intended Audience: The RAVLT and variations of it have been administered to people of ages 7 to 
the elderly. It has been used in various languages and cultures. Variations have been administered to 
individuals with hearing disabilities. Individuals must be alert and able to follow the test directions. They 
must have sufficient language skills to indicate their responses. 

Forms Available: The RAVLT has been modified extensively over the 40 years of widespread use. 
The most numerous modifications involve the memoiy items used in the test. The test was originally 
constructed in French and has been adapted for English, Hebrew, German, Italian, and Dutch. Many of 
these adaptations involved word changes rather than simple translations because of changes in syllable 
count. Another aspect that has been modified is the measurement methodology. The original method was 
memoiy recognition where the subjects were given a stoiy and asked to identify the memory items 
(words) in the stoiy. This was varied to use simple recognition from a list of words (either 30 or 50 
words), or recognition from a three item forced choice task. RAVLT's most common measurement 
method is free recall of memoiy items. Other variations include visual presentation where the subject 
must read the word aloud rather than have the word read to him or her, and variations in the feedback 
given to the subject (e.g., encouragement or identification of correct responses or repeated responses.) 

Other Tests by Rey: Rey Complex Figure Test and Recognition Trial (RCFT) A measurement tool for 
visiospatial ability and memoiy. 

Purpose and Recommended Use: The RAVLT is a simple memoiy assessment test. It can be used to 
test several aspects of memoiy such as working memoiy, long-term memoiy and learning ability. It has 
been used as a diagnostic tool to identify neurological, cognitive, and age-related disorders as well as a 
research tool to identify the manifestations of such disorders. In the process of establishing norms it has 
identified and quantified normal learning ability and memory ability as it changes with age. It was 
originally conceived for use with children but has more recently been used extensively with elderly 
populations. Its recommended use is in the assessment of memoiy capability with an emphasis on 
identifying memory deficits. It is recommended to be used as part of an overall assessment ensemble that 
can provide a comprehensive picture of the subject. 

Dimensions or Areas Measured: RAVLT can be used to measure working memoiy recall (Trials I-V 
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and Trial B), long-term memory recall (Trial VI and delayed trial), long-term memory recognition 
(recognition test - stoiy or list) and word learning ability (progress over Trials I-V). These measures 
combine to provide discriminative capability that is greater than the individual measures by themselves. 
For example, a lower than normal short-term and long-term memory score combined with a flat learning 
curve is an indicator of amnesia whereas low scores combined with a positive learning curve tend to 
indicate head trauma. 

Administration: The peculiar aspects of the test (i.e., those not included in most psychometric tests 
such as informed consent) include five consecutive learning trials of 15 words (Trials I-V). This is 
followed by one learning trial of an interference set of 15 different words (Trial B). A learning trial 
consists of the subject being presented 15 words at a rate of one per second. This presentation is normally 
provided by the test administrator reading the words aloud. Then the subject is asked to recall as many of 
the 15 words as they can remember in any order. Immediately following Trial B, the subject is asked to 
recall the original 15 words (Trial VI). Following this the subject is either presented with a list of words 
(e.g., 30 words where 15 are the original words and 15 are foils that may have semantic or phonemic 
similarity) or a stoiy containing the 1 5 words and asked to identify the words from the original list. This 
is the recognition test. A variation of the test includes a delay (20, 30, or 60 minutes) and another recall 
test. Subjects are not generally timed on any of the tests. 

Scoring Procedure: For each of the trials, the subjects responses can be scored as correct, repeated, 
repeated corrected (where the subject remembers that he or she has repeated the word), repeated 
questioned (where the subject asks if he or she has said that word before), error, error from the 
interference set, a frank confabulation, or phonemically or semantically similar error. These distinctions 
are not represented in the norm data however they can be useful in a more thorough assessment of the 
subject’s performance with regard to proactive and retroactive interference and malingering. If the subject 
constantly repeats a word without recognition of having done so or repeatedly asks if he or she has 
already mentioned a word, this could be a further indicator of memoiy deficits. For the recognition tests, 
a similar scoring can be used, namely correct, missed, error, or phonemically or semantically similar 
error. 

Description of Items: The memoiy items are all one or two syllable words that have a Thorndike - 
Lorge frequency (Thorndike & Lorge, 1972) of at least 50 per million, high imageiy and are at a third- 
grade reading level. Phonemic and semantic similarities are avoided. As mentioned above, there are 
several variations of the word lists. Many of these additional word lists were generated for test-retest 
assessment. Others were primarily for culture or language differences. 

Statistical Item Analysis: There has been no item analysis regarding the individual words in the 
RAVLT. I have not found any information regarding a more common occurrence of one word over 
another or the particular descriptive value of one word over another. However, there have been statistical 
comparisons of the different word lists that have been developed for the alternate forms of the test. I will 
discuss these between group analyses in the section on reliability. 

Method of Validation: Content validity is considered to be intuitively obvious given the nature of the 
test (its face validity) and it is not specifically addressed. What is more thoroughly addressed is 
concurrent validity both with other measures of memoiy and with discriminative tests on patients with 
known neurological disorders. Factor analyses have been used to confirm construct validity. 

Norm Groups: The norm data for the RAVLT is quite extensive, as it has been used for more than 40 
years. However, it is largely derived from a number of somewhat disparate studies. Schmidt (Schmidt, 
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1996) provides excellent tables of meta-norms based on these various studies. These meta-norms are 
based on n- weighted averages and standard deviations. They are stratified based on age. He provides 10 
categories of meta-norms: Age 13 (1 study, N=51), Ages 14-15 (2 studies, N=177), Ages 16-19 (3 
studies, N=78), Ages 20-29 (4 studies, N=498), Ages 30-39 (4 studies, N=1081), Ages 40-49 (5 studies, 
N=522), Ages 50-59 (4 studies, N=161), Ages 57-69 (4 studies, N=166), Ages 70-79 (4 studies, N=143) 
and Ages 76-89 (2 studies, N=50). In addition, he provides two general categories: Adult (4 studies, 
N=1989) and Elderly (2 studies, N=52). He provides a breakdown of each meta-norm that shows how 
each individual study compares to the meta-norm. 

The norm data includes means and standard deviations for responses for Trials I- VI, Trial B, the 
Delayed Recall Trial, the Recognition Memoiy Trial, and the sum score of Trials I-V. Since not all 
studies included the delayed recall and some studies used different methods for the recognition memoiy 
trial, Trials I-VI and Trial B are the most robust. 

Interpretation of Scores: While no cut off has been established for determining that a person’s 
memoiy is impaired, Schmidt (1996) suggests that a recall score of 8 or less on Trial V should arouse 
suspicion. The stratified norms show that the average recall on Trial V consistently decreases as age 
increases and the standard deviation consistently increases with age. The meta-norm mean for ages 76-89 
is 10 (SD=2.3). The meta-norm mean for ages 20-29 is 12.9 (SD=21.8). Therefore a score of 8 for Trial 
V for a 25 year old would be more suspicious than the same score for an 85 year old. While age is the 
dominant modifier, gender has also been shown to affect scores with females tending to have better scores 
than males. Some studies have shown effects from education and intelligence but not consistently. 

When interpreting the scores it is best to be specific if possible regarding the norm that is used. Many 
of the tests used to create the meta-norms used different procedures or word lists. The greater the 
similarity of the study used in the norm with the current study, the more comparable the data is to the 
norm. Schmidt (1996) provides the RAVLT performance data from several clinical groups such as 
patients with head injuries, learning disabilities, and Alzheimer’s disease. These can be used for 
comparison. 

Validity as Determined by Authors: Neither Rey nor Taylor appear to have produced significant 
validity studies. 

Validity as Determined by Others: Validity data from other researchers is plentiful. The various 
scores derived from the RAVLT (e.g., Trial VI and Trial I-V Total) have a strong and significant 
concurrence with Wechsler Memory Scale, the Wechsler Adult Intelligence Scale-R (WAIS-R) digit 
span, the Benton Visual Retention Test, the Visual Spatial Learning Test, and the California Verbal 
Learning Test. Factor analyses have show high single factor loadings on many of these tests when 
compared with RAVLT. RAVLT has been shown to discriminate between neurologically impaired 
individuals and normal individuals as well as being able to discriminate reliably on age. Factor analyses 
have shown 2 factor solutions interpreted as short-term (Trials I, II, and B) and long-term memory (Trials 
III, IV, V, and VI) and 3 factor solutions interpreted as acquisition (Trials I and B), storage (recognition 
memoiy trial), and recall (Trials V, VI, and delayed recall). 

Reliability: Reliability testing for memoiy tests can be difficult as there is a strong practice effect. 
One would want to use the exact same test for both test and retest. However, this would not be a valid 
descriptor of temporal reliability. One possible solution is to wait a sufficiently long period between test 
and retest to reduce practice effects. However, the practice effects in this case are rather robust and long 
time intervals can confound the reliability test, as memoiy scores will change with age. Alternate word 
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lists have been used to reduce this practice effect, however, it is not clear that the test is the same as the 
retest. Indeed, alternate word lists have been shown to have differences in difficulty according to average 
scores. However, it is generally agreed that these differences in difficulty, when appropriately 
counterbalanced, are so small that they do not provide any substantial reliability problems. 

Major Assumptions and Questions: The major assumption for this test is that memoiy span for words 
represents more general memoiy functions. The ability to use this test to discriminate between normal 
individuals and impaired individuals with diseases such as Alzheimer’s seems to bear this out. However, 
research in memory seems to indicate that there are several different types of memoiy processes (e.g., 
working memoiy, long-term memoiy, and primed memoiy (Endel Tulving, 2000)) and the amount of 
coverage provided by the RAVLT for working memoiy and priming remain to be seen. 

Distinguishing Characteristics: Perhaps the most distinguishing characteristic is the legacy of this test. 
It is tried and true and as with many standards, even if it did not measure what it purports to (which I 
believe it does) it would still be significant because it provides a reference for comparison. 

Desirable Features: The test is easy to administer, it is relatively fast, and it requires no special 
equipment. The norms that are available are excellent as are the instructions and the possibilities of 
alternate forms. Both reliability and validity have been demonstrated by those who do not have a vested 
interest in the test. 

Undesirable Features: While the norms are excellent, they have been compiled from several studies 
using different techniques. These variations may influence the outcome of the studies. This is probably so 
as some of the studies are in disagreement with each other. The RAVLT appears to have ceiling effects 
with some groups such as bright individuals, thus you cannot see the full range of the population. The 
samples used for most of the studies have higher than average education and IQ levels, which means that 
the norms may be too high for the population. There are no norms for many of the alternate forms. There 
has been no cross-cultural or cross-ethnic research using this test. 
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Appendix B 

Word list used for the experiment 


Five-letter words 


ABOUT 

COULD 

HEAVY 

PLACE 

SOUTH 

UNDER 

ABOVE 

COVER 

HORSE 

PLAIN 

SPEAK 

UNTIL 

AFTER 

CROSS 

HOUSE 

PLANT 

STAND 

VISIT 

AGAIN 

DEATH 

KNOWN 

POINT 

START 

VOICE 

ALONE 

DRESS 

LARGE 

POWER 

STATE 

WATCH 

ALONG 

DRINK 

LAUGH 

QUICK 

STONE 

WATER 

AMONG 

DRIVE 

LEARN 

RAISE 

STORY 

WHERE 

APPLE 

EARLY 

LEAVE 

REACH 

SWEET 

WHICH 

BEGIN 

EARTH 

LIGHT 

READY 

TABLE 

WHILE 

BEING 

EVERY 

MIGHT 

RIGHT 

THANK 

WHITE 

BLACK 

FIELD 

MONEY 

RIVER 

THEIR 

WHOLE 

BREAD 

FIRST 

MONTH 

ROUND 

THERE 

WOMAN 

BRING 

FLOOR 

NEVER 

SERVE 

THESE 

WORLD 

BUILD 

FOUND 

NIGHT 

SHALL 

THING 

WOULD 

CARRY 

FRESH 

NORTH 

SHORT 

THINK 

WRITE 

CAUSE 

FRONT 

OFTEN 

SIGHT 

THIRD 

YOUNG 

CHILD 

GREAT 

ORDER 

SINCE 

THOSE 


CLEAR 

GREEN 

OTHER 

SLEEP 

THREE 


CLOSE 

HAPPY 

PAPER 

SMALL 

TODAY 


COLOR 

HEART 

PIECE 

SOUND 

TRAIN 
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Appendix C 

Subject Consent Form 

RESEARCH SUBJECT INFORMATION AND CONSENT FORM 
TITLE: Assessing the effects of momentary distractions on memory retention. 

PROTOCOL NR: VCU IRB PROTOCOL NUMBER: 3762 
SPONSOR: Virginia Commonwealth University 
INVESTIGATOR: Paul C. Schutte 

This consent form may contain words that you do not understand. Please ask the investigator to explain any words 
or information that you do not clearly understand. You may take home an unsigned copy of this consent form to 
think about or discuss with family or friends before making your decision. 

PURPOSE OF THE STUDY: 

The purpose of this research study is to examine the effects of presenting briefly displayed words during memory 
retention task. The words are presented by briefly flashing them on the screen where you are currently viewing a list 
of words to be memorized. If you notice the word presentation, you should try to ignore it and concentrate on 
memorizing the words presented. 

You are being asked to participate in this study to help assess these effects. Your skills are not being tested. You 
should perform to the best of your abilities but you should not consider yourself to be under any pressure to perform. 

DESCRIPTION OF THE STUDY 

Your participation in this study will last up to 60 minutes. Approximately 80 subjects will participate in this study. 
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PROCEDURES 

This study will consist of two sessions. For each session, you will be asked to view and recall two lists of words 
(Lists 1 and 2 for the first session, Lists 3 and 4 for the second session.) There are 15 words in each list. These will 
be presented to you on a projection screen. 

You will be shown the first list of 15 words (List 1) on the screen one at a time at one-second 
intervals. You will then be asked to recall these words and write them down on a provided piece of paper 
that will be collected when you are finished. If you cannot remember a word, feel free to guess. The order 
of the words is not important. Once you have finished writing the words down you will turn the paper over. 
You will be shown these words in the same manner for two more trials. You will try to recall these words 
after each trial. Then you will be shown another set of 15 different words (List 2). During the presentation 
of the second set of words, you may notice a flicker of words on the screen. Please try to ignore them and 
concentrate on the new memory task (List 2). You will have to recall these new words (List 2) after they 
are presented to you. Finally you will be asked to recall the original set of 15 words (List 1). 

There will be a five-minute break. Then you will have another session with two new word sets. List 3 will be 
presented three times with a recall of the words after each presentation. Then List 4 will be presented followed by 
recall for List 4. Finally, you will be asked to recall List 3. 

After this session, there will be another five-minute break. Then you will be shown a list of 120 words. You will 
be asked to write a ‘1’ if the word was presented in List 1, ‘2’ if the word was presented in List 2, ‘3’ if the word 
was presented in List 3, or ‘4’ if the word was presented in List 4. Leave the space blank if you do not think that the 
word was previously presented. 

At the end of the test, you will be asked a number of questions regarding the test and words used in the test. 

In summary, your test will proceed in this order: 


Presentation of List 1 
Presentation of List 1 
Presentation of List 1 
Presentation of List 2 
Recall of List 1 

Recall of List 1 
Recall of List 1 
Recall of List 1 
Recall of List 2 

Break 

Presentation of List 3 
Presentation of List 3 
Presentation of List 3 
Presentation of List 4 
Recall of List 3 

Recall of List 3 
Recall of List 3 
Recall of List 3 
Recall of List 4 

Break 

Recognition test of 120 words. 


Final Questionnaire 
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RISKS AND DISCOMFORTS 

There are no known side effects. 

BENEFITS 

There are no benefits offered to you by your participation in this experiment other than the compensation 
offered by your instructor. 

COSTS 

There are no costs associated with this experiment. 

CONFIDENTIALITY 

Information from this study will be given to the sponsor. It may be given to other governmental agencies that 
are interested in this research. Records that identify you and the consent form signed by you may be looked at and/or 
copied for research or regulatory purposes by: 

• Virginia Commonwealth University 

Absolute confidentiality cannot be guaranteed because of the need to give information to these parties. The 
results of this research study may be presented at meetings or in publications. Your identity will not be disclosed in 
those presentations. 

COMPENSATION FOR INJURY 

In the event of physical and/or mental injury resulting from your participation in this research study, Virginia 
Commonwealth University and MCV Hospitals will not provide compensation. 

If you are injured or become ill as a result of participation in this study, contact the investigator immediately. 

VOLUNTARY PARTICIPATION AND WITHDRAWAL 

Your participation in this study is voluntary. You may decide to not participate in this study. If you do 
participate, you may freely withdraw from the study at any time. Your decision will not affect your standing or 
relationship with VCU. 

Your participation in this study may be stopped at any time by the investigator or the sponsor without your 
consent. The reasons might include: 

• the investigator thinks it necessary for your health or safety; 

• you have not followed study instructions; 

• there is an equipment malfunction 

• the sponsor has stopped the study; or 

• administrative reasons require your withdrawal. 
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QUESTIONS 

In the future, you may have questions about your study participation. If you have any questions, contact: 
Paul C. Schutte 
Phone: 757-864-2019 
Email: s2pcschu@ vcu.edu 

If you have questions about your rights as a research subject, you may contact: 

Office for Research Subjects Protection 
Biotech One 

800 East Leigh Street, Suite 114 
Richmond VA 23219 
Phone: (804) 828-0868 
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Do not sign this consent form unless you have had a chance to ask questions and have received satisfactory 
answers to all of your questions. 

CONSENT 

I have read this consent form. I understand the information about this study. All my questions about the study 
and my participation in it have been answered. I freely consent to participate in this research study. 

I understand that I will receive a signed and dated copy of this consent form for my records. 

By signing this consent form I have not waived any of the legal rights, which I otherwise would have as a 
subject in a research study. 

Subject Name, printed 
Subject Signature 

Date 

Signature of Person Conducting Consent Discussion 

Date 

Investigator Signature (if different from above) 

Date 
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Appendix D 

Word Recognition Sheet 


w 

w 


z 

; os i 

j 

Eh 

W 

B 

OS 

W 

X 

w 

W 

w 

z 

Q 

Q 

co 

w 

< 

H 

W l 

H 

H 
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u 

w 

OS 

u 


Eh 

►J 


3 


o 

OS 

Q 

ft 

Q ! 

ri 

CO 

H 

Eh 

Eh 

H 

H 

H 

H 

o 


os 

D 

B 

as 

O 

oS 



H 

o 

ri! 

C 

US 

a 

US 

US 

as 

o 

o 

O 

E-* 

Eh 

Eh 

EH 

B 1 


> 

> 

Z 

Z 

Z 

z 

Z 

S 

z 

z 

z 

5 


O 

>* 

C 

fU 

O) 


TD 

*5 

o 

§ 


w 

Pi 

■J Q 

B 

B 

Q 

Eh 

W 

W 

x 

Eh 

w 

B 05 

W 

W 

0 

B 

Q 

u 

w 


Eh 

< 

Z 

OS 

Eh 

z 

os 

W 

3 

Z H 

OS 

CO 

Z 

z 

OS 

z 

w 


D 

w 

< 


< 

o 

o 

W 

m 

< W 

w 

W 

H 

H 

H 

H 


S O 

O 

Pi 

Eh 

Eh 

Eh 

Eh 

EH 


c 

B B 

as 

US 

US 

US 

US 

CO 

co 

CO CO 

co 

co 

CO 

CO 

CO 

CO 

CO 

CO 

Eh 

EH EH 

Eh 

Eh 

Eh 

Eh 

Eh 


>u rememl 

ORDER 

OTHER 

PAPER 

PIECE 

PLACE 

PLAIN 

PLANT 

POINT 

POWER 

QUICK 

RAISE 

REACH 

READY 

RIGHT 

RIVER 

ROUND 

SERVE 

SHALL 

SHORT 

SIGHT 

>~ 

* 4 — 







X 


xtH^HWWjBzwfflzw 

PiOSt>COCOWU! 500 OS> 

PjrtlCPtJDWOOC^D^riJ 

^wwoosss*s<ww 

nsasasasasBBBJ^iJhSi 


EhEhJhEOSEhBS 
n m h w m :-h h 

ooss>o«eh 

HHOOWHOfa 

JgSSSSSO 


Q 

OS 

CO 

US 

CO 

B 

W 

3 

w 

CO 

EH 

CO 

Z 

> 

D 

> 

o 

ri! 

w 

H 

H 

O 

o 

OS 

w 

OS 

OS 

OS 

u 

u 

u 

Q 

Q 

Q 

Q 


W W W W W B 


OS 

o 

o 

B 


Eh W OS 

D > W 


z w o 0 w 
Z Z Z J 


eh 53 
< w 
w w 
os os 
0 0 


tfPOQjHWQOSWOS 

OOE-irtJOOOP^OHrtJWHHOSDHWO^ 

nmfcOjjsp^fxqwJososD^rfjaj^JO 

^rtJrtJrtJ^rtj^rtJCQPQCOPQmcQUUUUUU 
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Appendix E 

Post Test Questionnaire 

Subject #: 

Please circle the answer that best reflects your experience during this study. 
1 . The distractions were annoying 


1 

Strongly 

Agree 

2 

3 

Uncertain 

4 

5 

Strongly 

Disagree 

2. The distractions made it harder to remember the second set of words 

1 

2 

3 

4 

5 

Strongly 


Uncertain 


Strongly 

Agree 




Disagree 

3. 1 tried to read the distractions 



1 

2 

3 

4 

5 

Strongly 


Uncertain 


Strongly 

Agree 




Disagree 

4. 1 was able to read the distractions 



1 

2 

3 

4 

5 

Strongly 


Uncertain 


Strongly 

Agree 




Disagree 

5. 1 noticed that some of the distractions were previous words. 


1 

2 

3 

4 

5 

Strongly 


Uncertain 


Strongly 

Agree 




Disagree 

6. The repetition of previous words in the distractions helped me remember them. 

1 

2 

3 

4 

5 

Strongly 


Uncertain 


Strongly 

Agree 




Disagree 


7. On the back, please write down any memorization strategies you used during the experiment. 

Thank you for your participation! 
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Appendix F 

Preliminary Power Analysis 

One reason for using the RAVLT as a model for the experimental tasks in this experiment is the 
existence of established norms. These norms provide a predictive basis for estimating power in the 
experiment. The Meta-norms for ages 20 - 29 are listed in Table FI and are taken from (Schmidt, 1996). 
These represent the average number of words recalled out of 15 words. The RAVLT has five learning 
trials instead of three as in this experiment. Three was chosen for this experiment to reduce the likelihood 
of a ceiling effect (e.g., a large proportion of subjects remembering all 15 words). 


Trial 

Mean 

Standard Deviation 

1 

7.0 

1.8 

2 

9.9 

2.2 

3 

11.5 

2.1 

4 

12.4 

1.9 

5 

12.9 

1.8 

6 

11.5 

2.3 

Delayed Trial 

11.3 

2.5 


Table 7 Meta norms for the age group of 20 to 29 year-olds for the RAVLT 


After three learning trials, one would expect 95% of the subjects to fall between 7.3 (1 1.5 - 4.2) and 
15 (11.5 + 4.2) words recalled. After the secondaiy task, one might expect a reduction of about 1.5 in the 
mean. In this case, one would expect to see a mean of 10, with a span of 5 to 15. The standard deviation 
seems to increase after delay or interference and this is probably due to individual differences in memory 
ability. Hypothesis 1 predicts that the MMP session will not show as much of a decrease between trials 3 
and 4 as the other session. For the sake of the analysis, assume that the mean and standard deviation for 
the control-distraction session is 10 and 2.5, and that the MMP session is 11.5 and 2.1. How many 
subjects would be needed to achieve sufficient power of .80? 

Using the point biserial regression method: 


r pb=- 


(l 1 .5 -10) x .5 


2.5 


= .3 


f 

.91 


n = — + i + l= 78.5 + 2 = 81 
.1 


Thus the number of trials (that is, total sessions) required would be 81. Since each subject receives 
two sessions this requires a minimum number of subjects to be 41. The number of subjects was increased 
based on the results of a pretest. 
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